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Magued M. Bishay 



METHOD AND APPARATUS FOR LOW COST LINE-BASED 

VIDEO COMPRESSION 
OF DIGITAL VIDEO STREAM DATA 



BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to image processing and more 
specifically pertains to a video compression algorithm for increasing the data 
throughput on a limited bandwidth link between a host and a digital video camera. 

2. Description of Related Art 

A digital image represents a two-dimensional array of samples, where each 
sample is called a pixel. Precision detennines how many levels of intensity can be 
represented and is expressed as the number of bits/sample. Resolution of an 
image refers to its capability to reproduce fine details. Higher resolution requires 
more complex imaging systems to represent these images in real-time. In video 
systems, resolution refers to the number of line pairs resolved on the face of the 
display screen, expressed in cycles per picture height, or cycles per picture width. 
Full motion video is characterized with at least 24-Hz frames/sec, and 30 or even 
60 frames/sec for high definition TV. For animation, acceptable frame rate is in the 
range of 15-19 frames/sec while for video telephony it is 5-10 frames/sec. 
Videoconferencing and interactive multimedia applications require the rate of 15- 
30 frames/sec. 

Commercial imaging applications are surfacing in digital broadcast 
television, compact disk video, multimedia, video teleconferencing systems, 
dynamic medical imaging devices, and high definition TV. Imaging applications are 
time critical and computationally and data intensive, and require both the storage 
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and transmission of enormous data sets which can be accomplished with image 
compression. Achieving the compression ratios necessary for digital video involves 
the processing of individual images to remove spatial redundancies and a motion 
analysis of the sequence to remove temporal redundancies. 

5 The latest models of still and video cameras are solid-state cameras 

which use silicon chips for image data arrays. Since the size of the camera and 
the chip should be kept small, these cameras have to keep low the number of 
sampled image data transferred from the camera to an accompanying host 
image processing device. Furthermore, the mass consumers of the digital video 

10 cameras are used to viewing video images on TV at the frame rate of 30 frames 
per second. The currently available buses between digital video cameras and the 
host system have a limited bandwidth of less than 8 frames per second, which 
limits the number of video frames that can be transferred from the camera to the 
host and provides fuzzy pictures. These buses do not allow real-time video data 

15 transmission. 

In order to attract the average user to purchase a digital camera, the 
throughput of data through a low bandwidth transmission channel between the 
camera and the host has to be improved by compression. The real time image 
processing presently involves at least three types of tradeoffs: performance 

20 versus image resolution, performance versus storage size, and performance 
versus input and output bandwidth. The quality of performance is influenced by 
the cost. Expensive compression schemes, such as MPEG, are not cost 
effective for low cost single chip cameras. The basic measure for the 
performance of a compression algorithm is compression ratio, defined as a ratio 

25 of original data size and compressed data size. There is a trade-off between the 
compression ratio and the picture quality. Higher compression ratios may 
produce lower picture quality. Quality of compression can also vary according to 
source image characteristics and scene content. 
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Therefore, there is a need for a low cost method which can improve the 
data bandwidth between the digital video camera and the host by applying video 
compression to the digital video stream before the video frame is transmitted to the 
host computer, which is capable of generating sharp, high quality color images in 
5 real time. Furthermore, there is a need for small and inexpensive digital solid-state 
video cameras capable of transferring 30 video frames per second. These 
cameras have to have a low gate count and low power requirements so that the 
size and cost of the camera is minimal. 

OBJECTS AND SUMMARY OF THE INVENTION 

10 It is a primary object of the present invention to overcome the 

aforementioned shortcomings associated with the prior art and to provide an 
image processing device which uses a low cost video compression method which 
is capable of generating sharp, high quality color images in real time. 

Another object of the present invention is to provide a compression 

15 method which performs separate compression in the Y domain and the Cr/Cb 
domain. 

Yet another object of the present invention is to provide a compression 
method with minimum computational complexity which can be implemented in 
hardware. 

20 Still another object of the present invention is to provide small and 

inexpensive digital solid-state video cameras capable of transferring more than 
20 video frames per second, which have a low gate count and low power 
requirements. 

These as well as additional objects and advantages of the present 
25 invention are achieved by a video compression method that performs separate 
luminance (Y) domain compression of the video data on a line-by-line basis, 
without storing video data lines or video data frames, and separate chrominance 
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20 



(Cr/Cb) domain averaging of the video data on a region-by-region basis without 
storing video data in video frames. The Y and Cr/Cb domain compression steps 
are implemented in the digital solid-state imaging device hardware for real time 
link transmission of the compressed video data to the host computer. 
5 The method of the present invention is implemented in a pixel processing 

controller of the digital imaging device, such as a solid-state camera, working in 
isochronous traffic mode in the YCbCr 4:2:0 or 4:2:2 format, and adapted for a 
bandwidth-limited bus with isochronous pipes, one transmitting the Y domain 
values and another transmitting the Cr/Cb domain values. The Y domain 
10 compression module is adapted to detennine pixels in the video line which are 
tagged on a pixel-by-pixel basis, according to differences in the pixel luminance 
values, and lengths between the tagged pixels. The Cr/Cb domain compression 
module is adapted to calculate a single average value for a plurality of Cr 
locations, and a single average value for a plurality of Cb locations. 
15 In one embodiment of the present invention the Cr/Cb domain 

compression module obtains a single Cr value for each four Cr values, and a 
single Cb value for each sixteen Cb values in the 4:2:0 format, or a single Cr 
value for each eight Cr values, and a single Cb value for each thirty-two Cb 
values in the 4:2:2 format. In another embodiment of the present invention the 
compressed data is en coded and codes con catenated separately in th ^Y^ 
domain and in the ^^C^omain, before the transmission to the host computer. 
The concatenation in the Cr/Cb domain produces alternative Cr-only lines and ^^"^ 



Cr/Cb lines, where each Cr-only line has only Cr values, and each Cr/Cb line has 
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alternating Cr and Cb values. Ao 0 

25 BRIEF DESCRIPTION OF THE DRAWINGS D(ifti>\^^ - 

n 

The objects and features of the present invention, which are believed to 
be novel, are set forth with particularity in the appended claims. The present^^V^ 
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invention, both as to its organization and manner of operation, togetiier with 
further objects and advantages, may best be understood by reference to the 
following description, taken in connection with the accompanying drawings. 

Figure 1 is a schematic illustration of the positions of the Y,Cb,Cr sample 
5 sites on the scan lines of an interlaced picture, as used in the conventional 4:2:2 
YCbCr format of the prior art. 

Figure 2 is a schematic illustration of the sample site positions in a frame 
buffer, as used in the 4:2:2 YCbCr format of Figure 1 . 

Figure 3 is a schematic illustration of the positions of the Y,Cb,Cr sample 
10 sites on the scan lines of a non-interlaced picture, as used in the conventional 
4:2:0 YCbCr fomnat of the prior art. 

Figures 4A and 4B are schematic illustrations of the flow chart of a 
method embodiment of the present invention. 

Figure 5 is a schematic illustration of a digital video camera, a host, and a 
15 link between them, according to a preferred embodiment of the present 
invention. 

Figure 6 is a schematic illustration of the tagging scheme performed in the 
Y domain on a single active video line of a GIF size video frame, according to a 
preferred embodiment of the present invention. 
20 Figure 7 is a schematic illustration showing the Cr averaging scheme 

performed in the Cr/Cb domain, according to a preferred embodiment of the 
present invention. 

Figure 8 is a schematic illustration showing the Cb averaging scheme 
performed in the Cr/Cb domain, according to a preferred embodiment of the 
25 present invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The following description is provided to enable any person skilled in the art 
to make and use the invention and sets forth the best modes contemplated by the 
inventors of carrying out their invention. Various modifications, however, will 
5 remain readily apparent to those skilled in the art, since the general principles of 
the present invention have been defined herein specifically to provide a device and 
a method for maximizing the throughput of digitized video data on the bandwidth- 
limited link between a host and a digital video camera, which preferably use the 
YCbCr color system. 

10 The YCbCr color system was developed as part of Recommendation ITU- 

R BT.601 during the development of a woridwide digital component video 
standard. Y values are defined to have a nominal range of 16 to 235, and Cb 
and Cr values are defined to have a range of 16 to 240, with the 128 level 
corresponding to zero. There are several YCbCr sampling formats, such as 

15 4:4:4. 4:2:2, 4:1:1. and 4:2:0. 

Figure 1 illustrates the positions of the Y,Cb,Cr sample sites on the scan 
lines of an interiaced picture, as utilized in the 4:2:2 YCbCr format, often used in 
consumer video applications. In this system each sample has a Y, a Cb, and a 
Cr value. For every two horizontal Y samples, there is one Cb and one Cr 

20 sample. Each sample typically has 8 bits per component in consumer 
applications or 10 bits per component in editing applications. If a frame buffer, 
which has all three color components for each pixel, is used, each sample in 
consumer applications therefore requires 16 bits per pixel or 20 bits for editing 
applications, usually formatted as shown in Figure 2, where the pixel numbers 

25 are given after the dash line. For display, the Y samples for which no Cb and Cr 
data have been saved typically have to use interpolated Cb and Cr data from the 
previous and next samples that have the Cb and Cr data. 



-7- 



Figure 3 represents 4:2:0 coded picture sampling with the positions of the 
Y.Cb.Cr sample sites on the scan lines of a non-interlaced picture, as used in the 
4:2:0 YCbCr format, utilized by the H.261 and H.263 video teleconferencing 
standards and the MPEG video compression standard. The 4:2:0 YCbCr format 
5 implements a 2:1 reduction of Cb and Cr in both the vertical and horizontal 
directions. 

The video compression method implemented in the preferred embodiment 
of the present invention operates in the YCbCr 4:2:0 or 4:2:2 fomiat mode and is 
applied to the 4:2:0 or 4:2:2 data stream, although it may operate in other color 

10 systems and fomiat modes, as well. Because much better fidelity in the image can 
be preserved by not mixing together the color and brightness information, the 
method preferably operates on digital video cameras which have separate 
chrominance and luminance signals. Therefore the preferred method of the 
present invention operates in two separate domains, the Y (luminance) domain 

15 and the Cr/Cb (chrominance) domain. 

A flow chart of the preferred method of the present invention is shown in 
Figures 4A and 4B. The method operates on one video line at the time, and thus 
does not require lengthy frame buffers for its implementation. When applied in the 
Y domain the method operates in real-time and treats each horizontal video line as 

20 a separate entity, on a pixel-by-pixel basis. Therefore, in the Y domain there is no 
inter-frame or interline data correlation, because there is no need to save data 
before the transfer on a bus, and no line buffer is needed, thus saving space, cost 
and transfer time. The bus may be the Universal Serial Bus (USB) or another serial 
or parallel bus. The method is based on tagging pixels in a video line according to 

25 the differences in their luminance (Y) values. The value of a threshold for detecting 
a change in the luminance value between pixels can be predetermined and is 
programmable, as shown in step 13. The threshold value controls the image 
quality; the lower the threshold, the higher the image quality. In step 15 pixels' Y 
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vaiues are received one by one and pixels to be tagged are determined in step 17. 
The Y domain compression is perfonned in step 19. 

In the Cr/Cb domain an averaging step 21 is used to obtain the averaged Cr 
and Cb values for a transfer video block. Thus, in the Cr/Cb domain, a line buffer 

5 24, shown in Figure 5, is needed by the averaging step 21 and is used to hold one 
or, at most, two lines of digitized pixel data. In addition, to avoid streaking in the 
displayed image, the maximum size of a transfer field, containing pixel information, 
is limited in the step 13 to a finite and programmable value. 

The preferred method of the present invention is implemented within a 

10 digital video camera 10, attached to a host computer 28 via a bus 30, as shown in 
Figure 5. The host computer 28 is preferably a personal computer, capable of 
handling fast calculations and storing large amounts of data. The digital electronic 
camera 10 has an image sensor array 12, which may be covered with a color filter 
mosaic called a color filter array (CFA) 14. The method of the present invention 

15 may also be applied to monochrome cameras, in which case there is no Cr and Cb 
data. 

In the digital video camera 10 the analog signal coming from the image 
sensor array 12 is digitized using an analog-to-digital converter 16. The digital 
video camera 10 of the invention contains its own Pixel Processing Controller 

20 (PPC) 26, which is preferably an Application Specific Integrated Circuit (ASIC) 
chip, which has a Y domain compression module 20 for tagging the pixel locations, 
and a Cr/Cb domain compression module 22 for averaging the Cr and Cb values. 
The PPC 26 may also be used, if needed, to transfomri the image data from the 
camera's color system (such as RGB) to another color system, such as YCbCr, in 

25 which the luminance and the chrominance values are separated, and to perform 
compression in the 4:2:2 or 4:2:0 format. 

The PPC chip 26 has a limited-size on-board data storage area 29 to store 
a limited number of data values and parameters needed for the video 
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compression. The size of the on-board data storage area 29 is directly related to a 
predetermined value chosen in the step 13 for the maxinnum number of pixels 
allowed between the tagged pixels, which has to be kept small to obtain good 
resolution, and is preferably only 8 to 16 pixels. After the compression, the data is 

5 transferred to the host computer 28 which has a bus controller 34, a 
microprocessor 38, and a data storage area 36. The host computer 28 has to 
perform data decompression and data transformation from the YCbCr domain into 
the RGB domain before displaying the video frames on a display 40. 

The preferred method of the present invention uses a Y domain 

10 compression algorithm and a Cr/Cb domain compression algorithm. Figure 6 
illustrates how the Y domain tagging is performed on a single active video line of 
a CIF-size video frame, according to the preferred method of the present 
invention. The active pixels on the line are indicated as incoming pixels 1 through 
352, represented with the symbol Y,. Y^ represents the Y value for the previously 

15 tagged pixel. represents the length associated with the tagged pixel Y^. The 
variable L^has the value obtained with a counter of the step 15 and represents 
the number of pixels since the last tagged pixel. Y^^ represents the threshold 
value used for tagging, preferably saved with the Y^ , and values in the camera 
data storage area 29. 

20 The Y compression algorithm calculates the absolute value of the 

difference between an incoming pixel's Rvalue and the Y^ value of the previously 
tagged pixel in step 17. If the absolute value of the difference exceeds the 
threshold Y^h, preferably set by the host microprocessor 38, the Yj value for this 
incoming pixel and the number of pixels since the last tagged pixel (length are 

25 saved in the data storage area 29 of the camera PRC chip 26. This pixel 
becomes the new tagged pixel and the process is repeated for new pixels until 
the end of the current video line is reached. The tagged pixel luminance and 
length values are transmitted to the host computer 28 in step 50 where they are 
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used in a decompression and interpolation software program 60 stored in the 
host data storage area 36, to reconstruct the video, through replication and/or 
interpolation of the values between the tagged pixels on the video line. The 
process is repeated for the next video line. 

5 The host computer 28 receives the compressed video stream from the 

camera 10 and uses two algorithms in the interpolation software program 60 for 
decompressing and regenerating the original 4:2:0 or 4:2:2 image. These 
algorithms are applied to the Y domain and the Cr/Cb domain separately. 
Decompression in the Cr/Cb domain is a method opposite of the averaging 

10 method described below. In the Y domain, the decompression includes 
interpolation between the tagged pixels, which may be accomplished by copying 
a single pixel a number of times, based on the length value in the Y domain. 

Figure 6 illustrates a single active video line where blank circles represent 
incoming pixels Y, 63, chosen from a group of pixels Y^ - Y352, and shaded 

15 circles represent tagged pixels Yti, Yt2, 65. ^ and represent values of the 
length 67 associated with each tagged pixel 65. For example, the tagged pixel 
Yti has the length ^ of four because it has four pixels associated with it, while Yt2 
has the length Lt2 of three. 

The goal of the video compression method of the present invention is to 

20 minimize the number of tagged pixels transmitted to the host as well as to 
shorten the transfer field size. Therefore, if the length for each tagged pixel is 
kept beyond 16 pixels it can be represented with a field of 4 bits. In order to 
further increase the maximum length size, the length representing the number of 
pixels in a run associated with a tagged pixel may be encoded in the camera 10, 

25 after the step 19, and decoded in the host computer 28. In addition, the actual 
implementation of the Y domain compression method of the present invention 
may require use of special limitations, such as regarding the beginning and end 
of the video line. 
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Figures 7 and 8 show the averaging scheme for the Cr/Cb domain, used 
in an averaging step 21, and performed in the Cr/Cb compression module 22 of 
the digital camera PPC 26. In the Cr/Cb domain a single Cr value is calculated 
as the average value for preferably four Cr values in the 4:2:0 format, and for 

5 eight Cr values in the 4:2:2 format. A single Cb value is calculated to represent 
all Cb values in a group of pixels. In a block of 8x8 pixels there are preferably 
sixteen Cb values in the 4:2:0 format, and thirty-two Cb values in the 4:2:2 
format. Although the averaging reduces the precision of the color information, it 
is permissible in many types of video applications, as in video conferencing. If 

10 the data is in the 4:2:2 format, the 4:2:2 to 4:2:0 fomnat compression has to be 
performed in a method step 11 of Figure 4A on the Cr and Cb data. Y values do 
not change between the 4:2:2 and 4:2:0 formats. 

The algorithm of the step 21 performs averaging within each region of the 
Cr/Cb domain, and obtains a single Cr and a single Cb value for each region. In 

15 the 4:2:0 format, in the Cr domain this ratio is preferably 4:1, because four Cr 
values are averaged into a single Cr value, and in the Cb domain the ratio is 
preferably 16:1, because sixteen Cb values are averaged into a single Cb value. 
In the 4:2:2 format, in the Cr domain this ratio is preferably 8:1, because eight Cr 
values are averaged into a single Cr value, and in the Cb domain the ratio is 

20 preferably 32:1, because thirty-two Cb values are averaged into a single Cb 
value. Therefore, the Cr/Cb domain averaging algorithm of step 21 needs the 
short line buffer 24 to save data for this compression scheme. 

Figure 7 illustrates the Cr domain, 4:2:0 format, 4:1 averaging of the 
present invention, wherein each blank circle represents a Y value 81, a circle 

25 with an "x" symbol represents a Cr value 83, each dark circle represents a Cb 
value 85, and each circle with a symbol "-" represents an average Cr value 87. In 
this method, four adjacent Cr values are used to generated a single averaged Cr 
value 87, as shown in Figure 7. This calculated average Cr value 87 is sent to 
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the host computer 28 in the step 50 and is used to reconstruct the Cr values for 
this region of the video line. The rest of the Cr values for the video frame is 
calculated in the same manner. 

Figure 8 illustrates the Cb domain, 4:2:0 format. 16:1 averaging of the 

5 present invention wherein each blank circle represents a Y value 91, each dark 
circle represents a Cr value 93, a circle with a symbol "=" represents a Cb value 
95, each circle with a symbol represents a partial average Cb value 97, and 
each light-shaded circle represents an average Cb value 99. Four adjacent Cb 
values are used to generated a single intermediate averaged Cb value 97, as 

10 shown in Figure 8. This intermediate value 97 is averaged with three other 
values calculated similarly and the result is sent to the host computer 28 and 
used to reconstruct the Cb values for this region of the video line. The rest of the 
Cb values for the video frame are calculated in the same manner. 

The data obtained with the preferred method of the present invention may 

15 be further compressed by concatenating and/or encoding the information. For 
example, in the Y domain, because every tagged pixel in the image is preferably 
represented with an 8-bit Y value and the length with a 4-bit length value, the two 
4-bit length codes for two pixels may be concatenated into a single byte. A 
similar method may be used in the Cr/Cb domain, because the number of Cr and 

20 Cb values for every frame is constant and known, and there are 88 Cr values in 
each of 72 Cr lines per frame, and there are 44 Cb values in each of 36 Cb lines 
per frame. The compressed Cr/Cb format for a full video frame, according to the 
Cr/Cb compression algorithm of the present invention, may have the alternative 
Cr-only and Cr/Cb lines, where each Cr-only line has only Cr values, and is 

25 followed by the Cr/Cb line of alternating Cr and Cb values. 

The principles of the present invention have been proven by 
experimentation, using the preferred embodiments of the present invention and 
compared with the conventional video compression methods. The obtained 
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results have proven the feasibility and superiority of the present invention. The 
method of the invention was applied to a digital video camera with the USB bus 
working in an isochronous traffic mode. The USB bus is an isochronous bus 
which has two isochronous pipes with guaranteed, regularly scheduled, transfer 

5 of digital data. The bus pipes may be used to separately transmit compressed or 
uncompressed Y and Cr/Cb data, one domain per each pipe. 

The preferred method of the present invention has been applied to the 
USB-based digital video camera. The USB-based digital video camera typically 
contains an external buffer memory and may support three different modes: with 

10 no external buffer memory, with an external buffer memory of 32 Kbytes, and 
with an external buffer memory of 128 Kbytes. It has been confinned that the 
video compression algorithm of the present invention allows the camera to 
transfer a higher number of frames to the host than do conventional methods, 
depending on the compression ratio and the USB bandwidth available. 

15 Furthermore, the data transfer of CIF-size frames between the camera and the 
host has been performed with the higher rate even without using an external 
frame buffer, or when only a smaller, 32K byte frame buffer is used. 

The preferred method of the present invention is able to perform fast 
compression and decompression and enable real time transmission with a low 

20 average compression ratio of about 3:1 for moderately complex color images, as 
required for the USB bus, while preserving a high degree of image sharpness. 
With this method CIF-size video frames, having 352 pixels per line and 288 lines 
per frame, can be transmitted to the host at the average rate of between 20 and 
30 frames per second, with good resolution. It should be noted however that, as 

25 with the other video compression algorithms, the actual achieved compression 
ratio depends on the amount of detail and motion in the particular video frame 
sequence. An additional benefit of the present invention includes the feature that 
allows the change of the compression ratio with the threshold value Y^^. 
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Moreover, much higher compression ratios, such as 10:1, may be obtained when 
needed to squeeze image sequences through relatively low bandwidth 
transmission channels, such as for consumer delivery of moving images or when 
applied to applications that do not require very high quality, such as in video 
5 conferencing. 

In another aspect of the present invention a frame rate control algorithm is 
implemented, which works in conjunction with the compression algorithm and 
allows the user to request a specific frame rate. The host computer 28, with the 
knowledge of the available bus bandwidth and the size of the host data storage 
10 area 36, calculates the required number of bytes per frame to achieve the 
selected frame rate. The host computer 28 then calculates and sets the value of 
the maximum length size, as well as the Y^^ value, and sends these parameters 
to the camera 10. 

The prefered embodiments of the present invention are capable of 
15 producing digital imaging devices with a low gate count and power requirements, 
thus small, inexpensive, and yet capable of transferring 30 video frames per 
second. 

Although illustrated on a USB-based digital video camera, are believed to be 
applicable to other types of still and video cameras which use silicone chips, such 

20 as GIF cameras using a CCD detector. It is understood that the principles of this 
invention are particularly applicable to low cost digital imaging devices which have 
to perform on-the-fly video compression and transmit samples on a bus with the 
limited data bandwidth, which must perform data interpolation before the image is 
further processed and displayed. Such devices may be office automation digital 

25 linear sensors, digital fingerprint detectors, digital scanners, digital broadcast 
television, compact disk video, multimedia, video conferencing systems, desktop 
face-to-face visual communications (videophone), dynamic medical imaging 
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devices, high definition TV, video cassette recorders, copy machines, and fax 
machines. 

Those skilled in the art will appreciate that various adaptations and modifi- 
cations of the just-described preferred embodiment can be configured without 

5 departing from the scope and spirit of the invention. For instance, while the present 
invention has been described for the equipment using the USB bus, it is 
understood that the appropriately modified features of the present invention may 
be used to handle different buses and network channels, even wire optical fibers 
and broadcast channels. Furthemiore, although shown on digital video cameras 

10 using the YCbCr system and CIF-size fomiat, the present invention is equally 
applicable in the systems with other formats and color methods, for interlaced or 
non-interlaced picture. Therefore, it is to be understood that, within the scope of 
the appended claims, the invention may be practiced other than as specifically 
described herein. 
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CLAIMS 

What Is Claimed Is: 



1 1. A video compression method for maximizing a throughput of 

2 digitized video data on a link between a digital solid-state imaging device and a 

3 host computer, comprising the steps of: 

4 performing the luminance (Y) domain compression of the video data on a 

5 line-by-line basis without storing video data lines or video data frames; and 

6 performing the chrominance (Cr/Cb) domain averaging of the video data 

7 on a region-by-region basis without storing video data frames, 

8 wherein said Y and Cr/Cb domain compression steps are implemented in 

9 the digital solid-state imaging device hardware for real time link transmission of 

10 the compressed video data to the host computer. 

1 2. The video compression method of claim 1 wherein the link is a 



2 bandwidth-limited USB bus, and the digital solid-state imaging device is an USB- 

3 based camera which comprises a pixel processing controller with a low gate count, 

4 and is adapted to be small, inexpensive, and capable of transferring 30 video 

5 frames per second. 

1 3. The video compression method of claim 1 wherein the step of the Y 

2 domain compression comprises the following steps: 



3 determining a value of a threshold for detecting a change in the luminance 

4 value between pixels in a video line; 

5 tagging pixels in the video line on a pixel-by-pixei basis, according to 

6 differences in their luminance values, said tagging step comprising the following 

7 steps: 

8 calculating the absolute value of a difference between an incoming 

9 pixel luminance value and a previously tagged pixel luminance value, and 
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10 if the absolute value of the difference exceeds the threshold value, 

11 saving the incoming pixel luminance value as a new tagged pixel luminance 

12 value and the number of pixels since the previously tagged pixel as a 

13 length; and 

14 transmitting the new tagged pixel luminance value and length to the 

15 host computer. 

1 4. The video compression method of claim 1 further comprising a step 

2 of transforming the Cr/Cb data to YCbCr 4:2:0 or 4:2:2 fomriat, said step adapted 

3 to be performed before the Cr/Cb domain compression step. 

1 5. The video compression method of claim 1 wherein the step of the 

2 Cr/Cb domain compression comprises the following steps: 

3 calculating a single average value for a plurality of Cr locations; 

4 calculating a single average value for a plurality of Cb locations; and 

5 transmitting the average Cr and Cb values to the host computer. 

1 6. The video compression method of claim 5 wherein the step of 

2 calculating the single average value for the plurality of Cr locations comprises the 

3 step of obtaining a single Cr value for each four Cr values in the 4:2:0 fonnat, and 

4 obtaining a single Cr value for each eight Cr values in the 4:2:2 format. 

1 7. The video compression method of claim 5 wherein the step of 

2 calculating the single average value for the plurality of Cb locations comprises 

3 the step of obtaining a single Cb value for each sixteen Cb values in the 4:2:0 

4 format, and obtaining a single Cb value for each thirty-two Cb values in the 4:2:2 

5 format. 
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1 8. The video compression method of claim 7 wherein the step of 

2 calculating the single average value for the plurality of Cb locations comprises 

3 the following steps: 

4 averaging four-by-four adjacent Cb values from a region to generate a 

5 single intermediate average Cb value for each said four Cb values; and 

6 averaging the intermediate average Cb values to obtain a single average 

7 Cb value. 

1 9. The video compression method of claim 1 wherein the digital solid- 



2 state imaging device is chosen from a group which consists of office automation 

3 digital linear sensors, digital fingerprint detectors, digital scanners, digital broadcast 

4 television, compact disk video, multimedia, video teleconferencing systems, 

5 dynamic medical imaging devices, high definition TV, video cassette recorders, 

6 copy machines, and fax machines, and said bus is chosen from a group which 

7 consists of computer buses, network channels, wire optical fibers, and broadcast 

8 channels. 



1 10. The video compression method of claim 1 further comprising the 

2 steps of: 

3 encoding the compressed data with a minimum number of bits; and 

4 concatenating the bits of the codes representing the compressed data, 

5 wherein said concatenation is performed separately in the Y domain and in the 

6 Cr/Cb domain, and the encoding and concatenating steps are performed before 

7 the transmission to the host computer. 

1 11. The video compression method of claim 1 wherein said step of 

2 concatenation in the Cr/Cb domain produces alternative Cr-only lines and Cr/Cb 

3 lines, where each Cr-only line has only Cr values, and each Cr/Cb line has 

4 alternating Cr and Cb values. 
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1 12. In a digital video imaging device which works in the YCbCr 4:2:0 or 

2 4:2:2 format and is attached to a host computer via a link, a pixel processing 

3 controller comprising: 

4 a Y domain compression module for tagging pixel locations in the Y domain 

5 according to a predetemnined criteria; 

6 a Cr/Cb domain compression module for averaging the Cr and the Cb 

7 values in the Cr/Cb domain; 

8 in the Y domain a predetermined threshold value for detecting a change in 

9 the luminance value between pixels in a video line; and 

10 in the Y domain a predetermined value for maximum number of pixels 

1 1 allowed between the tagged pixels. 

1 13. The controller of claim 12 wherein the link is a bandwidth-limited 

2 USB bus, the digital video imaging device is a solid-state USB-based camera, and 

3 the pixel processing controller has a low gate count, and is adapted to be small, 

4 inexpensive and capable of transferring 30 video frames per second. 

1 14. The controller of claim 12 wherein the link is a bandwidth-limited bus 

2 with isochronous pipes, the digital video imaging device is a solid-state camera 

3 working in isochronous traffic mode, and each bus pipe transmits one domain per 

4 pipe. 

1 1 5. The controller of claim 12 wherein the predetermined threshold value 

2 and the predetemnined value for maximum number of pixels allowed between the 

3 tagged pixels are supplied by the host computer. 

1 16. The controller of claim 12 wherein: 

2 the Y domain compression module is adapted to determine pixels in a video 

3 line which should be tagged on a pixel-by-pixel basis, according to the 

4 predetermined criteria based on the differences in pixel luminance values; and 
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5 the imaging device transmits the tagged pixel luminance values and lengths 

6 in the video line between the tagged pixels to the host computer. 

1 17. The controller of claim 12 wherein the Cr/Cb domain compression 

2 module is adapted to calculate a single average value for a plurality of Cr locations 

3 and a single average value for a plurality of Cb locations, the compressed data are 

4 encoded with a minimum number of bits, and the bits of the codes representing the 

5 compressed data are concatenated separately in the Y domain and in the Cr/Cb 

6 domain, before the transmission to the host computer. 

1 18. The controller of claim 12 wherein the Cr/Cb domain compression 

2 module is adapted to obtain a single Cr value for each four Cr values, and a single 

3 Cb value for each sixteen Cb values in the 4:2:0 format, and a single Cr value for 

4 each eight Cr values, and a single Cb value for each thirty-two Cb values in the 

5 4:2:2 format. 



1 19. A digital imaging device attached to a host computer via a link, 

2 comprising: 

3 an image sensor anray for obtaining luminance (Y) and chrominance (Cr, 

4 Cb) values of pixels in a video line; and 

5 a pixel processing controller having: 

6 a Y domain compression module for tagging pixel locations in the Y 

7 domain, and 

8 a Cr/Cb domain compression module for averaging the Cr and the 

9 Cb values in the Cr/Cb domain. 
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1 20. The digital imaging device of claim 19 wherein said link is a 

2 bandwidth-limited bus with isochronous pipes, wherein a first bus pipe transmits 

3 the Y domain values and a second bus pipe transmits the Cr/Cb domain values, 

4 and the digital imaging device is a solid-state camera working in isochronous traffic 

5 mode in the YCbCr 4:2:0 or 4:2:2 format. 

1 21 . The digital imaging device of claim 20 wherein: 

2 the Y domain compression module is adapted to determine pixels in the 

3 video line which are tagged on a pixel-by-pixel basis, according to differences in 

4 the pixel luminance values; 

5 the camera transmits to the host computer the tagged pixel luminance 

6 values and lengths between the tagged pixels; and 

7 the Cr/Cb domain compression module is adapted to calculate a single 

8 average value for a plurality of Cr locations, and a single average value for a 

9 plurality of Cb locations. 

1 22. The digital imaging device of claim 21 wherein: 

2 the Cr/Cb domain compression module is adapted to obtain a single Cr 

3 value for each four Cr values, and a single Cb value for each sixteen Cb values 

4 in the 4:2:0 format, and a single Cr value for each eight Cr values, and a single 

5 Cb value for each thirty-two Cb values in the 4:2:2 format; 

6 said compressed data are encoded and codes are concatenated 

7 separately in the Y domain and in the Cr/Cb domain, before the transmission to 

8 the host computer; and 

9 said concatenation in the Cr/Cb domain produces alternative Cr-only lines 

10 and Cr/Cb lines, where each Cr-only line has only Cr values, and each Cr/Cb line 

11 has alternating Cr and Cb values. 
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ABSTRACT OF THE DISCLOSURE 

A video compression method is provided and implemented in a pixel 
processing controller of a digital solid-state imaging device for maximizing 
throughput of digitized video data on a link between the digital solid-state 

5 imaging device and a host computer. The method performs separate luminance 
(Y) domain compression of the video data on a line-by-line basis, without storing 
video data lines or video data frames, and separate chrominance (Cr/Cb) domain 
averaging of the video data on a region-by-region basis without storing video 
data in video frames. The Y and Cr/Cb domain compression steps are 

10 implemented in the digital solid-state imaging device hardware for real time link 
transmission of the compressed video data to the host computer. 



Price, Gess & Ubell 



JOSEPH W.PRICE 

SxiND UBELL * JViV^Jlf, XJi^UkJ %^ ^ M,M^^^ A PROFESSIONAL CORPORATION 

^a,™ ATTORNEYSATIAW TEL™ (7U)26..^ 

x;nfnTAT:T T \4nFFATT FACSIMILE: (714)261-9072 

^SE G^Tm 2100 S.E. MAIN STREET, SUTTE 250 facsimile (7I4)261-i726 

BRADLEY D BLANCHE* 

SANDRAM.PARKER IRVINE. CALIFORNIA 92614^238 Pg^'ie<«»"«'» 

♦ADM. VA ONLY 



COMBINED DECLARATION AND POWER OF ATTORNEY 



Supplicant: BEHNAM S. KATIBIAN, et al. 

ttor: METHOD AND APPARATUS FOR LOW-COST LINE-BASED VIDEO 
m COMPRESSION OF DIGITAL VIDEO STREAM DATA 

lAttorneys' Docket: ROC1-BC20/97RSS423 

i EXPRESS MAIL LABEL NO. EM322760409US 



DATE OF DEPOSIT; March 13, 1998 



Docket No. ROC1 -BC20/97RSS423 



DECLARATION FOR PATENT APPLICATION 
AND POWER OF ATTORNEY 

As a below named Inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an 
original, first, and joint inventor (if plural names are listed below) of the subject matter which is 
claimed and for which a patent is sought on the invention entitled Method And Apparatus For 
Low Cost Line-Based Video Compression Of Digital Video Stream Data, the specification 
of which 

(check one) M is attached hereto □ was filed on 

as Application Serial No. 
and was amended on (if applicable). 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the examination of this 
application in accordance with Title 37, Code of Federal Regulations, § 1.56(a). 

I hereby claim the benefit under Title 35, United States Code, § 120 of any Unjted States 
application(s) listed below and, insofar as the subject matter of each of the claims of this 
application is not disclosed in the prior United States application in the manner provided by the 
first paragraph of Title 35, United States Code, §112, I acknowledge the duty to disclose 
material information as defined in Title 37. Code of Federal Regulations, § 1.56(a) which 
occurred between the filing date of the prior application and the national or PCT international 
filing date of this application: 



(Application Serial No.) (Filing Date) (Status patented, pending, abandoned) 



(Application Serial No.) (Filing Date) (Status patented, pending, abandoned) 

I hereby appoint the following attomey(s) and/or agent(s) to prosecute this application and to 
transact all business in the Patent and Trademari< Office connected therewith: 

William C. Cray, Reg. No. 27,627; Phillip K. Yu, Reg. No. 35,742; Susie H. Oh, 
Reg. No. 36,391; Joseph W. Price, Reg. No. 25,124; Albin H. Gess, Reg. No. 
25,726; Franklin D. Ubell, Reg. No. 27.009; Michael J. Moffatt, Reg. No. 39,304; 
Doyle B. Johnson, Reg. No. 39,240, Bradley D. Blanche, Reg. No. 38,387, and 
Sandra M. Partner, Reg. No. 36,233, with full power of substitution and 
revocation hereby given to William C. Cray. 

Address all telephone calls to Albin H. Gess at telephone no. (714) 261-8433 

Address all conrespondence to Albin H. Gess 

Price, Gess & Ubell 

2100 S.E. Main Street, Suite 250 

In/ine, Califomia 92614-6238 



Page 1 of 3 



Docket No. ROC1-BC20/97RSS423 



DECLARATION FOR PATENT APPLICATION 
AND POWER OF ATTORNEY 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on infomnation and belief are believed to be tme; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Full name of first 
joint inventor 

Inventor's Signature 

Residence 

Post Office Address 



Magued M. Bishay 




1411-1/2 Cffff Drive 

Newport Beach, California 92663 



1411-1/2 Cliff Drive 

Newport Beach, California 92663 



Date March 12, 1998 



Citizenship Egypt 



Full name of second 
joint inventor 

Inventor's Signature 

Residence 



Behnam S. Katibian 



1375 Kelton Avenue, No. 207 
Los Angeles, Califomia 90024 



Date March 12, 1998 



Citizenship U.S.A. 



Post Office Address 



1375 Kelton Avenue, No. 207 
Los Angeles, Califomia 90024 



Full name of third 

joint inventor 

Inventor's Signature Date 

Residence Citizenship 

Post Office Address 



Page 2 of 3 



JOSEPH W PRICE 
ALBINKGESS 
FRANKLIN D.UBELL 
DOYLE B. JOHNSON 
MICHAEL J. MOFFATT 
GORDON E. GRAY m 
BRADLEY D. BLANCHE* 
SANDRA M. PARKER 



Price, Gess & Ubell 

ATTORNEYS AT LAW 

2100 S.E. MAIN STREET, SUTTE 250 
IRVINE, CALIFORNIA 92614^238 



A PROFESSIONAL CORPORATION 
TELEPHONE: (714)261-8433 
FACSIMILE: (714)261-9072 
FACSIMILE (714)261-1726 

e-mail: pgu@ix.netcom.com 



*ADM. VAONLY 



DRAWING COVER SHEET 



JApplicant: BEHNAM S. KATIBL«^, et al. 



FFor: METHOD AND APPARATUS FOR LOW-COST LINE-BASED VIDEO 
COMPRESSION OF DIGITAL VIDEO STREAM DATA 



Attorneys' Docket: ROC1-BC20/97RSS423 



EXPRESS MAIL LABEL NO. EM322760409US 



DATE OF DEPOSIT; March 13, 1998 




9i$ — ® ♦ ® • ® 

3 »^ ® 9 ® 0- 

® Y.C8.CR SAMPIE 
• Y$AMPL5CNLY 









PtXEt 


Pixa 


PiXSL 


0 


t 




3 * 


4 


5 


Y7-0 


Y7-1 


Y7«2 


Y70 


Y7.4 


Y7.5 


Ye-0 


v«.i 


Y6.2 


Ye*3 


Y(-« 


Ya*5 


' Y30 


Y5.1 


YS-2 


Yd-3 


Y3.< 


Y5.5 


Y4-0 


Y4-t 


Y4.2 


YX*3 


Y4.4 


Y4.5 


Y3-0 


Y3M 


Y3-2 


Y3-3 






Y2»Q . 


Y2-f 




Y2-3 


Y2-4 


Y2-5 


Y1»0 


YJ«1 ' 


' Y1.2 


Y1«3 


Yl*4 


Yt-S 


YO.O 


YO-1 


Y0«2 


Y0--3 


Y0.4 


YO-S 


€67*0 




C87*2 


CK7»2 


C87*4 


CR7.4 


CSd-0 




C86*2 


cns*2 


C65-4 


ca5«4 


CB5>0 




C85-2 


CR5*2 


CSS -4 








ce4.2 


efu-2. 


.C84-4 


CR4*4 




Xft3-0 


C83*2 


cn-a 


Cd3-4 


CR3-4 




Cft2*0 


CK«2 




CW.4 


CR2*4 


1 eai>o. 




C81*2 


CK1*2 


CS1«4 




1 CB0*O 


C^*0 


C80.2 


eA0*2 


C90*4 


CR0«4 



1 ^ 


® 




® 






® 




® 











^♦^TQL uBnur ?»'V^ 






''i K JLi "JLi ii. ^ Jis 

ooo^4oo#oooo --C 




DOOOOOOO 

D]8foOOgpO 
®®®0®®® 

nooooood 



Figure 7 



